Spectrum-Based Fault Localization without Test Oracles

Spectrum-Based Fault Localization (SBFL) is one of the most promising approaches towards fault localization, and has received a lot of attention due to its simplicity and effectiveness. It utilizes various program spectra and the associated testing result of each individual test case, namely failed or passed, to evaluate the risk of containing a fault for each program entity with different statistical formulas. However, it suffers from a crucial problem which makes it infeasible in many application domains, that is, its assumption of the existence of test oracle. In practice, there are many programs in various domains without such oracles, where obviously, the existing SBFL techniques cannot be applied at all. To address this problem, we introduce a novel type of slices, metamorphic slice (mslice), which is property based, as different from the traditional test case based slices in SBFL. We propose to use mslice instead of slice, and its associated metamorphic testing (MT) result of a metamorphic test group, which is either violated or non-violated, rather than the testing result of failed or passed for individual test cases, to evaluate the risk for each program entity of being faulty. In this way, we can extend SBFL to application domains without test oracles, needless to say that our proposed method is still applicable if the application domains have test oracles. We use a popular UNIX utility program, grep, in our case study to evaluate the effectiveness of the proposed method. With three popular risk formulas (Orchiai, Jaccard and Tarantula), we compare the effectiveness of using mslice and traditional slice in SBFL. The empirical results suggest that for test suites of the same size, there is no significant difference between mslices and traditional slices.